WHAT IS CLAIMED IS: 



oi ^ A user \inter face apparatus comprising: 

a first Wnsor attached to a first portion of a 
body of a user; 
5 a second seizor attached to a second portion 

different from the Xf^st portion; 

means for generaJting action information of the 
second portion on the b^sis of a relative position of 
the second portion with ir^spect to the first portion, 
10 which is detected by said ^sirst and second sensors; and 
determination means fo\ determining a user 
instruction corresponding to tT;ie generated action 
information . 

2. The apparatus according to ^laim 1, wherein the 
15 first portion is a head. 

3. The apparatus according to cla\m 1, wherein the 
second portion is a hand. 

4. The apparatus according to claim wherein said 
first sensor detects a location/posture qf the first 

20 portion. 

5. The apparatus according to claim 1, wherein said 
second sensor detects a location and posture Npf the 
second portion, 

6. The apparatus according to claim 1, whereiS^ the 
25 action information includes information which per\:ains 

to a state change of the second portion with respeat to 



a locatVon of the first portion, and information which 
pertainsNto a location change velocity of the state 
change . \ 

7. The apparatus according to claim 1, wherein the 
action informatsLon includes information which pertains 
to a state changev of the second portion, with respect to 
a location or location/posture of the first portion, 
and information whicnv pertains to a location change 
acceleration of the state change. 

8. The apparatus according to claim 1, wherein the 
action information includes^ information which pertains 
to a posture of the second portion with respect to a 
posture of the first portion. \ 

9. The apparatus according toXclaim 1, wherein the 
action information includes informa.tion which pertains 
to a moving direction of a location \>f the second 
portion with respect to a posture of rhe first portion. 

10. The apparatus according to claim J!y further 
comprising: \ 

means for storing a value of the relative 
position of the second portion with respect tio the 
first portion, and a plurality of state valuesXwhich 
are defined in advance as a result of transitioiV of the 
value; and \ 
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meapis for storing a plurality of different user 
instruction^ values corresponding to the plurality of 
state value s\ 

11. The apparatus according to claim 1, wherein said 
determination me\ns decomposes the determined user 
instruction into aS^plurality of instruction operands, 
and outputs the operands. 

12. The apparatus according to claim 1, wherein when 
said generation means determines that a relative 
relationship between a looation/posture of a head 
detected by said first sensm:, and a location/posture 
of a hand detected by said seesond sensor indicates an 
action of the user whose line o*f sight is pointing to a 
predetermined portion of the hanos, 

said determination means outputs a user 
instruction for outputting an operat\on guidance. 

13. The apparatus according to claiitXl, further 
comprising a third sensor for detecting\a bent angle of 
a finger. \ 

"^^^ A game apparatus comprising: \ 

a first sensor for detecting a locatioWposture 

of a head of a player; \ 

a second sensor for detecting a locat ion/©osture 

of a hand or arm; \ 

means for estimating an action of the piayerXon 

the basis of a relative location/posture of the hanM or 



arm ^ith respect to the location/posture of the head, 
which \re detected by said first and second sensors; 
and 

mean^ for outputting a player command 
correspondings^^ to the estimated action. 

15. The apparatus according to claim 14, further 
comprising display means for displaying an image of a 
game scene in front\of the head of the player for the 
player . 

16. The apparatus acc6j:ding to claim 14, wherein said 
display means is a head-mounted display. 

17. The apparatus accordino to claim 14, wherein 
three steps including preparatsion, execution, and 
return steps, are prepared for fshe player command, and 
the steps are respectively defineoyas: 

a preparation action step forXthe player command 
when the location of the hand of the player moves to a 
position behind a position in front of \ face of the 
player; 



an execution step for the player comiA^nd when the 
location of the hand of the player moves forward from a 
rear position after the preparation action; am 

a return action step for the player comman>d when 
the location of the hand of the player returns to\a 
position of the face of the player after the forwar\ 
movement . 
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18). A \user interface method for outputting a user 
instructicm to a predetermined apparatus or program, 
comprising : 

the ste^of detecting a location of a first 
portion of a bodv of a user and a location of a second 
portion dif f erent \f rom the first portion using first 
and second sensors ^a1:tached to the user; and 

the step of determining a user instruction on the 
basis of a relative posdNiion of the second portion with 
respect to the first portion, which are detected by 
said first and second sensors, and outputting the 
determined user instruction toVthe apparatus or program. 

19. The method according to cl^im 18, wherein said 
first and second sensors respectively detect 
locations/postures of the first andXsecond portions. 

20. The method according to claim lot, wherein the 
first portion is a head of the user, an<^ the second 
portion is a hand of the user, 

21. The method according to claim 18, further 



A 



comprising the step of detecting information which 
pertains to a state change of the second port\i.on with 
respect to a location or a location/posture of\the 
first portion, and information which pertains to\a 
location change velocity of the state change. 
22. The method according to claim 18, further 
comprising the step of detecting information which 
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peVtains to a state change of the second portion with 
respQCt to a location or a location/posture of the 
first ppi^tion, and information which pertains to a 
locationNchange acceleration of the state change. 

23. The method according to claim 18, further 
comprising tn^^step of detecting a posture of the 
second portion with respect to a posture of the first 
portion . 

24. The method according to claim 18, further 
comprising the step ofv detecting a moving direction of 
a location of the Seconal portion with respect to a 
posture of the first portron. 

25. The method according t\) claim 18, further' 
comprising the step of storingVa value of the relative 
position of the second portion vrith respect to the 
first portion, and a plurality of user instructions 
which are defined in advance as a re?ult of transition 
of the value, 

26. The method according to claim 18,\wherein the 
output step includes the step of decomposing the 
determined user instruction into a pluralit^y of 
instruction operands, and outputting the operands. 

27. The method according to claim 18, whereiX when it 
is determined that a relative relationship between a 
location/posture of a head detected by said firstX 
sensor, and a location/posture of a hand detected bV 
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saidVsecond sensor indicates an action of the user 
whose JNine of sight is pointing to a predetermined 
portion of the hand, 

the output step includes the step of outputting a 
5 user instructNion for outputting an operation guidance. 

28. The method according to claim 19,. further 
comprising the step of detecting a bent angle of a 
finger from a third, sensor. 

29. A computer reaoable storage medium, which stores 
10 a program of a user interface method for outputting a 

user instruction to a predetermined apparatus or 
program, storing: 

a program step of det^>pting a location of a first 

□ 

rJ portion of a body of a user akd a location of a second 

i u \ 

nj 15 portion different from the firs\ portion using first 

and second sensors attached to the user; and 

a program step of determining a user instruction 
on the basis of a relative position \pf the second 
portion with respect to the first portion, which are 
20 detected by said first and second sens\^, and 

outputting the determined user instruction to the 
apparatus or program. 

30. A game apparatus for displaying a^^^G^i\nage in 
front of a field of view of a player, comprising: 

25 a first sensor for detecting a location a 

first portion of a body of the player; 
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second sensor for detecting a location of a 
second portion of the player, which is different from 
the first pjortion; and 

game pi^gress means for proceeding with a game by 
determining a command the player wants to input on the 
basis of a relatJVe position of the second portion with 
respect to the first portion, which are detected by 
said first and second\sensors , and executing the 
determined command. 

31. The apparatus according to claim 30, wherein said 
first and second sensors res^pect ively detect 
locations/postures of the firsst and second portions. 

32. The apparatus according tOv claim 30, wherein the 
first portion is a head of the player, and the second 
portion is a hand of the player. 

33. The apparatus according to clai^n 30, further 
comprising means for detecting information which 
pertains to a state change of the secondXportion with 



respect to the location or a location/posture of the 



first portion, and information which pertains to a 
location change velocity of the state change, \and 
wherein the command is generated in furtlr^r 
consideration of the detected velocity. 
34. The apparatus according to claim 30, furtherN 
comprising means for detecting information which 
pertains to a state change of the second portion with\ 
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respect to the location or a location/posture of the 

\ 

first portion, and information which pertains to a 



location change acceleration of the state change, and 

wherein the command is generated in further 
considerat\pn of the detected acceleration. 
35. The apparatus according to claim 30, further 
comprising mearrs for detecting a posture of the second 
portion with respfect to a posture of the first portion, 
and 




10 wherein the command is generated in further 

consideration of the detracted posture of the second 
portion . 

36. The apparatus according to claim 30, further 
Q \ 
lU comprising means for detecting\a moving direction of 

i'U \ 

fU 15 the location of the second portron with respect to a 

\ 

Q posture of the first portion, and Vherein the command 

is generated in further consideration of the detected 

\ 

moving direction of the location of the second portion. 

\ 

37. The apparatus according to claim 30, further 

20 comprising a memory for storing a value ^'f^ the relative 
position of the second portion with respect to the 
first portion, and a plurality of player commands which 
are defined in advance as a result of transiti^pn of the 
value . 

25 38. The apparatus according to claim 30, wherein said 
game progress means decomposes the determined player 
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command into a plurality of operands, and executes the 
commandXin accordance with the plurality of decomposed 
operands . 

39. The apparatus according to claim 33, wherein when 
a relative resJ.ationship between a location/posture of a 
head detected bo/ said first sensor, and. a 
location/postureNpf a hand detected by said second 
sensor indicates that a line of sight of the player is 
pointing to a predetermined portion of the- hand, 

said game progress means generates a player 
command for outputting ari operation guidance. 

40. The apparatus according to claim 30, further 
comprising a third sensor fos^ detecting a bent angle of 
a finger 

41. The apparatus according to claim 1, wherein the 
action information further includes geometric 
information of the first and second^ portions in a 
reference coordinate system. 

42. The apparatus according to claims 14, wherein said 



im*^ 
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estimation means extracts geometric information of the 
first and second portions in a reference ^coordinate 
system, and estimates a player action on the basis of 
the geometric information. 

43. The method according to claim 18, wherein the 
detection step includes the step of extracting 
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geometric information of the first and second portions 
in a r\^erence coordinate system. 

44. Tbe apparatus according to claim 30, wherein said 
game progress means further extracts geometric 

inf ormat ion \of the first and second portions in a 
reference coordinate system, estimates an action of the 
player on the ba'^s of the geometric information, and 
determines the command the player wants to input on the 
basis of the estimarfed action. 

45. An interface apparatus comprising: 

a first sensor f o^r\detect ing a location or motion 
of a first portion of a boGj^ of a user; 

a second sensor for detecting a location or 
motion of a second portion of izhe user, which is 

\ 

different from the first portion^ 

\ 

determination means for analyzing outputs from 
said first and second sensors and determining a command 



input by the user on the basis of patsterns of the 
outputs from said first and second sensors; and 

output means for outputting the command 
determined by said determination means to\an object to 
be controlled. 
46. The apparatus according to claim 45, wherein the 
first portion is a head, and the second portion is a 
hand . 



47. The apparatus according to claim 45, wherein said 
first anov second sensors respectively detect locations 
and postures of the first and second portions. 

48. The apparatus according to claim 47, wherein said 
first and second sensors respectively detect location 
change velocities^, location change accelerations, and 
moving directions the first and second portions. 

49. The apparatus according to claim 48, wherein said 



determination means anaUyzes an action of the user 
10 using information of a change in location, the location 
Q change velocity, location change acceleration, and 

relative moving direction ofXthe second portion with 

iir. respect to the first portion, and specifies a command 

□ 

pj corresponding to the action . 



15 50. The apparatus according to c^laim 49, wherein said 
determination means has storage info.rmation which 
defines in advance a relationship between the action of 
the user and the corresponding command 1\ 

51. The apparatus according to claim 50, wherein the 
20 command output from said determination means is 

\ 

supplied to the object to be controlled, and an image 
corresponding to the command is displayed. 
52- A user interface method comprising: 

the first detection step of detectinq a lo^cation 

\ 

25 or motion of a first portion of a body of a user; 
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^he second detection step of detecting a location 

\ 

or motion of a second portion of the user, which is 

\ 

different \from the first portion; 

the determination step of analyzing outputs of 
5 the first and^econd detection steps and determining a 
command input by\ the user on the basis of patterns of 
the outputs of the\ first and second detection steps; 
and 

the output step ^of outputting the command 
10 determined in the determination step to an object to be 
controlled. 

53. The method according to claim 52, wherein the 
first portion is a head, and the second portion is a 



ry hand. 

rij 15 54. The method according to claim 52, wherein the 

□ X 

I'l^ first and second detection steps include the steps of 

respectively detecting locations and\postures of the 
first and second portions. 

55. The method according to claim 54, \ wherein the 
20 first and second detection steps includeVthe steps of 

respectively detecting location change vei'ocities , 
location change accelerations, and moving directions of 
the first and second portions. 

56. The method according to claim 55, whereir^ the 
25 determination step includes the step of analyzing an 

action of the user using information of a changeXin 
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location, the location change velocity, location change 
acceleration, andVtrelative moving direction of the 
second portion withXrespect to the first portion, and 
specifying a command cs;orresponding to the action. 

57. The method according to claim 56, wherein the 
determination step includes the step of. determining the 
command on the basis of storage information which 
defines in advance a relations\ip between the action of 
the user and the corresponding command. 

58. The method according to claJlm 57, wherein the 
object to be controlled displays an r^jiage corresponding 
to the command. 
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